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Processing images or audio representations 

Field of the Invention 

5 The present invention relates to processing of images that are 
displayed by representation means to a viewer thereof based on 
image data stored in a data unit. The present invention 
relates also to processing of audio presentations replayed by 
representation means to a listener thereof based on audio data 
10 stored in a data unit. In particular, but not exclusively, the 
invention relates to representation of edited or otherwise 
adjusted versions of images or audio presentations. 

Background of the Invention 

15 

An image can be represented by means of an appropriate 
representation device. For example, an image can be displayed 
by a display device comprising a display screen or other means 
for presenting an image based on image data. An audio 
20 presentation, such as speech or a musical or other artistic 
audio presentation, can be represented by an appropriate 
device comprising sound reproduction means, such as a 
loudspeaker . 

25 The image data can be created, for example, by means of a 
digital camera. When a photograph is taken by the digital 
camera, the digital camera creates an image data file that may 
be stored in a memory means of the camera. The image data may 
also or alternatively be stored in a removable data carrier, 

30 such as in a tape, CD (compact disc) or a memory card. The 

same principles apply to recordings of audio representations. 
Digital cameras or digital audio recording devices are well 
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known in the art, and will thus not be explained in more 
detail herein. 

Typically the image or audio data is stored in a file, record 
5 or similar data unit. It is possible to compress the data 
stored in the data unit. The compressed data requires less 
storage capacity when stored in an appropriate data storage 
means. A reason for the compression is that the digital 
cameras or other portable devices are typically provided only 
10 with a limited storage capacity. Furthermore, if the data unit 
is to be transported between two or more devices, compressed 
data files require less transportation capacity from the 
communication media between the devices. 

15 An example of compressed data units is a compressed JPEG file 
for storing image data. The term v JPEG ' stands for Joint 
Photographic Experts Group which is formed by experts 
nominated by standards bodies and major companies. A standard 
by the JPEG is IS 10918-1 (ITU-T T.81), which is the first of 

20 a multi-part set of standards for still image compression. 

JPEG is based on the features of this standard, in association 
with a JFIF file format by C-Cube Microsystems. 

Digital cameras typically compress images with a so called 
25 lossy compress algorithm before storing these to a memory. The 
above referenced JPEG supports the lossy compression. By means 
of the lossy compression the size of the image files can be 
made small enough so that it is possible to store the image 
data in limited storage capacity of the camera. A disadvantage 
30 of the lossy compression technique is that it loses 
information during the compressing operations. 
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One important feature of the digital photography is that it is 
possible to adjust and edit the images afterwards. The changes 
can be made by anybody. That is, the creator (i.e. the 
photographer) or the viewer or any other person in a chain 
5 between the creator and the viewer may edit or otherwise alter 
the image. Common tasks that may be done to an image are, for 
example, adjustment of brightness and contrast, white balance 

H or gamma curve of the image. These operations are not 

El computationally expensive and relatively easy to perform. 

JJ:J 10 Another important editing task is cropping of the image. In 
the cropping unnecessary or otherwise unwanted areas of the 

Jjl image are cut out from the boundary areas of the image. In 

that way image size resolution, and thus size of the image 

pj file can be reduced. 

m 15 

C3 An unwanted side effect of the editing processes is that the 

quality of the image may be reduced. For example, if an image 
is edited and stored successive times the quality of image may 
be reduced in each editing cycle comprising decompression - 
20 compression and storing. This is caused by the possibility of 
loosing information during the image compression stages, 
especially if a lossy compression algorithm is used. 

Thus the visual quality of the image may be reduced every time 
25 the image is fetched from, decompressed, compressed and stored 
again in the image data file. Since the decompression- 
compression cycle typically happens every time the image data 
is processed, the visual quality of the image may get 
progressively worse each time an image is subjected to 
30 modifications. The image may start gradually look worse and/or 
it starts include artifacts. 
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Furthermore, e.g. the JPEG file format does not contain any 
indication regarding the resolution in which the image has 
been saved. The image processing softwares typically allow the 
user to select the resolution in which the image is to be 
5 saved. Thus, if the image is edited and subsequently saved 

with lower resolution the quality of the image gets worse. On 
the other hand, saving the image with higher resolution than 
^ what has been used previously may also be disadvantageous. The 

M additional data comprises error that was caused by the 

yj 

m 10 previous compression cycle and the size of the image file is 
:f Z unnecessarily increased. In addition, different softwares may 

Rj use different quality parameters (e.g. 0-1, 0%-100%, 0-255 and 

13 



15 In the above images are edited and stored in a lossy 

compression format. Another possibility is to store images by 
so called lossless compression formats. JPEG2000 is an example 
of a compression standard supporting both lossy and lossless 
compression. However, the lossless formats require more memory 

20 capacity for the image data storing than the lossy formats. 
Therefore the lossless formats may not be suitable for 
substantially small portable devices, such as for example 
digital cameras or display devices or portable digital audio 
recorders or players. 

25 

Furthermore, data may be lost even in the "lossless" 
compression. For example, lets assume that an audio signal 
gets values between 0 and 1. If the signal is amplified some 
of the audio data may become "clipped" because a part of the 
30 signal may have been amplified out of its dynamic range. Thus 
a part of the audio information may become lost and the 
quality of the representation becomes worse. It shall be 
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appreciated that amplification of an audio signal is analogous 
to the adjustment of brightness of an image. 

The digital imaging is gaining more popularity. An especially 
5 rapidly growing area of digital image handling is believed to 
be transportation of digital image data over data networks. At 
least the transmitting device or the receiving device may be a 
portable user equipment or other station capable of 
communication over a wireless interface with the data network 
10 apparatus. As the size and/or weight is typically an important 
feature of portable devices, the above discussed limitations 
in size and memory capacity need to be taken into 
consideration when designing such devices. 

15 The end users of the image capturing or displaying devices may 
also wish to edit the images. To enable this it is likely that 
even the user equipment will be provided with at least some 
level of adjustment facility. The inventor believes also that 
at least in the first wireless image data transmission devices 

20 the image compression methods are likely to be based on lossy 
compression techniques. This is believed to be so because of 
the pixel count in the image and demands on storing capacity 
grow faster than technologies that may be used to increase the 
storage capacity. However, the deteriorating quality of the 

25 images may become a problem also with devices that are based 
on the lossless techniques. 

Summary of the Invention 

30 Embodiments of the present invention aim to address one or 
several of the above problems. 
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According to one aspect of the present invention, there is 
provided a data unit for storage of image or audio 
representation data so that an image or an audio presentation 
can be represented by representation means based on said data, 
5 the data unit comprising information regarding adjustments 
that have been made to said image or audio representation 
after said data was input in the data unit. 

According to another aspect of the present invention there is 
10 provided a display device, comprising a display means, a 

storage means for storing image data associated with an image, 
said image data being included in an image data unit, and a 
processor means for processing image data adapted to process 
image data based on information also included in said image 
15 data unit, said information being indicative of changes to be 
made to the image data before the image is displayed on the 
display means. 

According to another aspect of the present invention there is 
20 provided a method of displaying an image, comprising: storing 
image data associated with the image in a data storage means; 
storing information indicative of modifications made to the 
image in the data storage means so that said information can 
be fetched when the image is to be displayed by a display 
25 device; modifying the image based on said information; and 
displaying the modified version of the image. 

The embodiments may enable edition and/or adjustment of images 
or audio representations which are stored in a compressed or 
30 otherwise processed format without any substantial reduction 
in the quality when the image or audio representation is 
represented. The embodiments may enable more efficient use of 
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the storage capacity of the user equipment adapted to process 
the data. The embodiments may provide an easy way to undo 
changes and return to the original image or audio presentation 
without losses in quality or information content. An image can 
5 be adjusted in several different ways without affecting the 
original image information. For example, the embodiments may 
enable storage of two different white balance or 
brightness/contrast settings and selection one of these based 
on the conditions or situations in which the image is 
10 displayed. 

Brief Description of Drawings 

For better understanding of the present invention, and ways to 
15 put the invention into effect, reference will now be made by 
way of example to the accompanying drawings in which: 

Figure 1 shows an image presentation means in which 
embodiments of the present invention may be utilised; 
Figures 2a to 2c illustrate an embodiment of the 
20 invention; 

Figure 3 is a flowchart illustrating the operation of an 
embodiment of the present invention; and 

Figures 4a and 4b illustrate a further embodiment of the 
present invention. 

25 

Description of Preferred Embodiments of the Invention 

Reference is made to Figure 1 which shows a portable user 
equipment 1 comprising a display means 2. More particularly, 
30 the user equipment 1 comprises a mobile station capable of 

communication over a data communication network via a wireless 
interface between antenna means 7 and the network. Elements of 
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the data communication network are not shown for clarity 
reasons . 

The operation of the representation device 1 and/or the 
5 display screen 2 may be controlled by a control processor 5. 
The device 1 is also preferably provided with data storage 
means 4. An operational connection 6 is provided between the 
data storage means 4 and the control processor 5 . 

10 The display means 2 is adapted to present images based on 

image data. The image data may be stored in the storage 4. The 
image data may be processed by the processor 6 before 
displaying the image to the user. The processing may be 
accomplished based on a control software run on the processor. 
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The device 1 may comprise one or more control buttons 3 . The 
control buttons may be used to control the operation of the 
device and also to edit the image before and during the 
display thereof. 



The schematic exemplifying image displayed by the display 
screen 2 illustrates two boxes and a ball. The display means 
may comprise any appropriate display arrangement, such as a 
liquid crystal display (LCD) . It shall be appreciated that the 

25 display means 2 of Figure 1 is only an example, and that the 
display means 1 does not form an essential part of the 
invention. The display means may alternatively comprise e.g. a 
printer device producing a reprint of the image based on image 
data processed by the user device 1. Since the presentation 

30 devices and possible display means as such are well known by 

the skilled person, these will not be described in more detail 
herein . 
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The following will discuss in more detail exemplifying 
embodiments of the invention. More particularly, the firstly 
described embodiment relates to prevention of quality losses 
5 when adjusting colours (brightness/contrast, white balance, 
gamma correction and so on) of the image. 

In the first example described with reference to Figures 2a to 
2C, a JPEG file format is used for storing and/or transporting 
10 image data. JPEG file format is a well known format in the art 
of storing images using lossy compression, and the principles 
thereof will thus not be explained in more detail. For the 
purposes of understanding the embodiment it is sufficient to 
note that a JPEG file is provided with a comment field. 

15 

The inventor has realised that it is possible to store free 
form information that associates with the adjustments in such 
a separate field of the data unit contained compressed data. 
In accordance with a preferred embodiment the information can 
20 be stored in a predefined manner in the comment field of the 
JPEG file. 

When the image is edited the data stored in the comment field 
can be edited accordingly. That is, the data associated with 

25 the changes in the image can be stored to the JPEG file by 

storing this data in the comment field instead of amending the 
actual image data. By means of this it may be possible to 
avoid subsequent decompress -compress steps of the image data 
and thus quality losses in the image itself. This procedure 

30 may also quicken the saving operations of the image since it 
is not necessary to recompress (i.e. repack) the image after 
the processing thereof. 
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This operation is illustrated in more detail by Figures 2a to 
2c and by the flowchart of Figure 3. As shown by the Figures, 
a JPEG file 20 contains two separate data fields 21,22. The 
5 first field 21 is for compressed image data of an image and 
the second field 22 is for comments that associate with the 
image . 

Figure 2a illustrates situation when image data is stored in 
10 the image data field 21 for the first time. The comment field 
22 comprises first settings for the brightness and contrast of 

1 the image. These may be the initial settings that were set 

«" when the image data was created as a basic setting to be used 

m when displaying the image. It shall be appreciated that while 

■i is? 

2 15 Figures 2a to 2c show two data fields for clarity reasons, an 
O image data file may comprise more than two data fields. 

Figure 2b shows the situation after the image data has been 
fetched once and the viewer of the image has adjusted the 
20 brightness and contrast parameters to suit better for his/hers 
use. Figure 2c shows the situation after n edition cycle. 

In the prior art arrangements the image data in the image data 
field 21 would have been changed according to the changes in 
25 the brightness and contrast parameters after each edition 

cycle, that is after the amendments shown by Figures 2b and 
2c. Thus the image data saved in field 21 after these editing 
cycles would not have been any longer the sama data as what it 
initially was in Figure 2a. 
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In the preferred embodiment of the present invention the image 
data saved in the image data field 21 remains substantially 
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the same after each editing cycle. Instead of amending the 
image data stored on field 21 when editing the image, the 
original image data is processed based on the information in 
the comment field 22 before displaying the image. The 
5 processing is accomplished based on the comment field 22 so 
that the image to be displayed to the viewer corresponds the 
latest changes while the changes are done to the original, 
unchanged image data stored in the image data field 21. 

10 For example, the processor 5 of the display device 1 may be 

adapted to process the image data stored in on field 21 based 
on the information in the comment field 22. In order to be 
able to display the latest version of the image, image 
adjustment operations indicated by the comment field 

15 information are done to the decompressed image data before the 
image is displayed by the display screen 2. 

A further embodiment relates to prevention of quality losses 
when adjusting image boundaries, i.e. during the so called 

20 image cropping. This embodiment is described with reference to 
Figures 4a and 4b. As above the data is stored in a JPEG file 
format. The original image 30 has been divided into smaller 
image data blocks 31. The image data blocks 31 may comprise, 
for example, 8 pixel * 8 pixel blocks. Lossy JPEG compression 

25 may be used for compressing each of the image data pixel 
blocks 31 one by one to the compressed format. In the 
compressed format the blocks 31 are preferably stored in the 
same order as they are in the original image 30. 
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It is sometimes necessary to cut of the border areas of the 
image 30, i.e. to crop the image. For example, it may be 
necessary to adjust the boundaries of the image, e.g. so that 
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the image fits the available display screen area of the 
display device. The cropped area of the image is shown to 
comprise the six blocks 31 within the rectangular window 32. 

5 By means of the invention it is possible to crop a desired 
image area without first decompressing all of the cropped 
pixel blocks 31 for preview purposes. Only selected pixel 
block can be fetched from the image data file based on 
information associated with the image data. The information 
10 regarding the blocks that need to be selected may be stored in 
the comment field of the JPEG file as described above. 

In Figure 4a the selected blocks are the ones located within 
the cropping window 32. The blocks outside the window 32 are 

15 not processed, i.e. not are not fetched nor subjected to any 
decompression operations. The cropping may be done without 
affecting the original JPEG image data on the image data file. 
The cropped pixel blocks may copied to a new, different JPEG 
file. The image data associated to image 32 of Figure 4b may 

20 be stored after viewing for later use or it may deleted. 

Thus it is possible to avoid a decompression-compression step 
of the image data during the cropping procedure and thus avoid 
quality losses caused by a decompression-compression cycle, as 

25 a re-compression step can be avoided. By providing cropping 
information in the comment field modifications of the image 
data itself may be avoided. The cropping information may be 
used to define different image area borders for the same basic 
image so that it may be used in different occasions. The 

30 embodiment may also save processing time and/or capacity since 
it is not necessary to decompress all data blocks before the 
selection of the blocks to be fetched for the display. 
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If a cropped image is fetched and saved based on the 
compressed data blocks and cropping information, a new image 
comprising the fetched blocks may be saved without first 
5 decompressing and compressing the image data. In the above 
example that would means that the selected 8*8 pixel blocks 
are saved as a new image data filed file comprising 8*8 pixels 
blocks. If the blocks are decompressed, it is not necessary to 
save them in similar 8*8 blocks, but the blocks may be 
10 subjected to changes. 

The above has described usage of the comment field of a JPEG 
file. This does not does not prevent viewing an image based on 
image data stored in the JPEG file by means of programs and 



CO 15 display applications which are not capable of understanding 

13 

J the information content in the comment field. The comment 

field is a part of the JPEG standard, and the JPEG compatible 
devices are typically adapted to ignore any information in the 
comment field they cannot interpret and/or utilise. There is 
20 no limitations for the content that may be stored in said 
comment field. 

The above example refers to information associated with 
processing of features such as the colours, brightness, 
25 contrast, white balance and/or boundaries of an image. 

However, it is also possible to store information associated 
with other editing operations. 

•j 

For example, an image may be subjected to a sharpen operation. 
30 The sharpen operation is used to emphasise edges that are 

present in an image. If too much sharpening has been provided 
the visual quality of the image may suffer. If too strong 
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sharpening is removed data will in most cases be lost. A 
mention of any sharpening operations added in the comment 
field may be used for the adjustments later on without 
modifications to the actual image data. In addition, if the 
5 image data is saved before any sharpening operations and the 
information thereof is added in the comment field, the 
compression of the image data may be easier to perform by the 
U JPEG since the image data contains less high frequencies. 

;f' ;:;; i: 

W 10 Another further example comprises information associated with 

Zl the quality parameter that has been used when the image was 

^ saved. By adding an indication regarding the quality and the 

a quality parameter in which the image was saved in the comment 

If! field it is possible to use the same quality in all subsequent 
15 saving operations. 
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It shall be appreciated that although the embodiments of the 
present invention have been described in the context of a JPEG 
format, the invention is also applicable to any other 
20 compression and/or processing techniques. For example, it is 
possible to store image data in a JPEG 2000 file or any other 
data format which supports a field enabling inclusion of 
additional data associated with the image. 

25 It shall be appreciated that whilst the Figure 1 embodiment of 
the present invention was described in relation to a mobile 
station, embodiments of the present invention are applicable 
to any other suitable type of user equipment capable of 
presenting an image or audio presentation to the user thereof. 

30 These devices include, without limiting to these, digital 

cameras and recorders, various display devices provided with 
image editing facility and so on. 



The term image refers also to an image of a plurality of 
images that are displayed in succession. For example, moving 
pictures or multimedia presentations consist of a series of 
successive images. Each data unit that associates with an 
image (or several images) may be provided with information 
regarding adjustments that have been made to the presentation. 
In such case the adjustments are made to each image (or 
picture frame) after the associated image data has been 
fetched from the memory means. Alternatively, at least one of 
the data units is provided with said adjustment information. 
The display is then adjusted based on said at least one data 
unit containing adjustment information. The same technique can 
be analogously applied to audio presentations. 

The way how the information associated with the changes in the 
image or audio presentation is stored in the separate data 
field may be standardised, at least to some level, so that 
different software applications that write/read information in 
the separate data field may recognise the information 
associated with the amendments. 

It is also noted herein that while the above describes 
exemplifying embodiments of the invention, there are several 
variations and modifications which may be made to the 
disclosed solution without departing from the scope of the 
present invention as defined in the appended claims. 



